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Abstract 

William Stephenson specifically developed Q methodology, or Q, as a means of measuring 
subjectivity. Q has been used to determine perspectives / views in a wide variety of fields from 
marketing research to political science but less frequently in education. In higher education, the 
author has used Q methodology to determine views about a variety of situations, from students' 
views about a newly developed bioinformatics course to faculty members' views of reading circles 
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as a professional development experience to improve teaching and learning in their classrooms. 
The purpose of this article is to introduce Q methodology and demonstrate its versatility in 
addressing research purposes in higher education, especially where the focus is on determining 
student or faculty perceptions about a topic. Such determinations can be helpful for program 
evaluation and improved teaching/learning in higher education. 


Determining Faculty and Student Views: Applications of Q Methodology in Higher Education 

The purpose of this article is to introduce Q methodology and demonstrate its versatility in 
addressing research purposes in higher education, especially where the focus is on determining 
people's perceptions and / or grouping people based upon their views. William Stephenson 
specifically developed Q methodology, or Q, as a means of measuring subjectivity (Brown, 1980, 
2008; McKeown & Thomas, 1988; Stephenson, 1953], Q has been used to determine perspectives / 
views in a wide variety of fields from marketing research to political science (Brown, 1980; 
McKeown & Thomas, 1988] but less frequently in education (Brown, 1980], In higher education, 
the author has used Q methodology to determine views about a variety of situations, from students' 
views about a newly developed bioinformatics course (Ramlo, McConnell, Duan, & Moore, 2008] to 
faculty members' views of reading circles as a professional development experience to improve 
teaching and learning in their classrooms (Ramlo & McConnell, 2008], 

A Q study begins with the selections of items to be sorted. These items, typically statements 
related to the topic, often come from qualitative beginnings such as focus groups and interviews 
(McKeown & Thomas, 1988; I. Newman & Ramlo, 2010], These items are then sorted by 
participants as they provide their perspectives by sorting these items into a grid, typically with a 


range such as +5 (most like my view] to -5 (most unlike my view]. The participants judge each item 
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relative to the others as they place them into the grid and rearrange as they desire (Brown, 1980; 
McKeown & Thomas, 1988; Stephenson, 1953], The sorts are then entered into specialized software 
for analysis, such as PQ Method (Schmolck, 2002], which produces a variety of informative tables 
based upon factor analysis results (Brown, 1980; I. Newman & Ramlo, 2010; Schmolck, 2002], In Q, 
the sorts are factor analyzed such that people with similar views are grouped into factors. Thus, 
each factor represents a view about the topic (Brown, 1980; McKeown & Thomas, 1988; 

Stephenson, 1953], Factors are interpreted based upon the tables produced from the analyses of 
the Q sorts (Brown, 1980; I. Newman & Ramlo, 2010], More information on the process of Q 
methodology follows along with examples from a variety of studies to assist in describing the 
application of Q methodology in higher education. We will begin with the Q-sample and the Q-sort. 


The Q-sample 

The items are derived from various means including interviews and focus groups (Brown, 
1980; McKeown & Thomas, 1988; I. Newman & Ramlo, 2010], The items for the Q sort may consist 
of anything from statements to pictures (Brown, 1980; McKeown & Thomas, 1988; Stephenson, 
1953], In some studies, the statements are created by the researcher or someone involved with the 
project based upon program goals or other criteria such as with the evaluation of a new 
bioinformatics course (Ramlo et al., 2008]or of a faculty reading circle program designed to 
improve faculty's teaching (Ramlo & McConnell, 2008], An earlier study by Ramlo (2005] 
developed 50 items for sorting by having faculty use a think-pair-share (Lyman, 1992] exercise 
early within the discussion about the creation of a School of Technology on a university campus 
(Ramlo, 2005], In that Ramlo (2005] study, participants wrote down two weaknesses, two 
strengths, and any other concerns about the proposed creation of a School of Technology at a large, 
public university. Individuals then broke into groups of two or three to discuss what they had 
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written which then led to a large group discussion. This method led to 50 unique statements for the 
participants to later sort. 


The Q-sort 

Items, such as statements, are typically placed on individual strips of paper. Instructions for 
the Q sort typically begin by asking the participants to place the statements, as they read them, into 
one of three piles based on the condition of instruction: (1] More Like My View (2) Neutral (3] 
Least like my view about the topic. It is helpful if participants can make these piles relatively equal 
in preparation for the final sort. Once the three piles are created, participants distribute the 
statements into a grid such as the one shown in Figure 1. Because the items are on individual strips 
of paper, participants can re-arrange them until they are satisfied that their placements match the 
participant's view. It is important to note that because participants judge each statement relative to 
the others based upon their own views, each sort represents the participant's subjectivity about the 
topic. Because participants interpret each statement, operational definitions and validity are not 
concerns in Q methodology (Brown, 1980], 
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Figure 1 A Q-sort grid showing the number of statements to be placed within the range from -4 
(most unlike my view] to +4 (most like my view]. 


The analyses 

In Q, the sorts are factor analyzed such that people with similar views are grouped into 
factors. Each factor represents a view about the topic (Brown, 1980; McKeown & Thomas, 1988; 
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Stephenson, 1953], The analyses produce a number of descriptive outputs that are interpreted to 
confirm or explore people's perspectives. Thus, Q methodology shares many of the focuses of 
qualitative research while utilizing the type of statistical analyses typically found in quantitative 
studies. In this way, Q methodology fits into the paradigm of mixed methods research (I. Newman & 
Ramlo, 2010; Stenner & Stainton-Rogers, 2004], As Bazeley (2010] suggests, the integration of 
qualitative and quantitative research into mixed methods allows the researcher to produce findings 
that are of greater use and better address the research purpose (I. Newman, Ridenour, Newman, & 
DeMarco, George Mario Paul Jr., 2003], However, the mixing of qualitative and quantitative 
methods affects interpretations of research quality (Greene, 2008], For instance, differences in 
statistical considerations related to grouping people with factor analysis with 
subjective/qualitative data (Q sorts] versus objective data (Likert surveys] are discussed elsewhere 
(Stephenson & Burt, 1939], As an example, however, in Q methodology frequently there are 
misinterpretations about sample size. In Q the sample size is represented by the number of 
statements sorted, not the number of participants sorting the statements. Thus in Q methodology 
researchers are concerned about having sufficient number and types of statements to represent the 
communicability of the topic (Brown, 1980], The number of participants sorting is determined by 
the purpose of the study (Brown, 1980; I. Newman & Ramlo, 2010; Stephenson, 1953], 


three Q studies in higher education 

Three distinct studies, each with a different purpose and population, are discussed within 
this section. Because it is not practical to discuss the results of three studies in higher education in 
detail, the results and conclusions from several studies are briefly presented within this paper. Each 
of these studies is published thus readers may find details elsewhere. The purpose of this article is 
to demonstrate the versatility of Q methodology within higher education. With this in mind, the 
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author has selected three studies to profile here: Evaluating a new course in bioinformatics; 
creating a new school of technology; and investigating students' views of learning physics. The 
study about investigating students' views of learning physics is discussed first. Only this study 
includes data tables and this is done to help the reader understand the types of information 
reported within Q and to help interpret the discussions about the subsequent studies contained 
within the article. 

Investigating students' views of learning physics 

Our first example is one that focuses on curriculum, teaching, and learning in a first 

semester physics course for engineering technology majors, both associate and bachelor degree 
level, at a large Midwestern public university. Considerable research and curriculum development 
has focused on students' learning of force and motion concepts yet research shows that many 
students fail to gain Newtonian-based understanding of force and motion concepts(Ramlo, 2008c; 
Redish, Saul, & Steinberg, 2000; Thornton & Sokoloff, 1998], Research has demonstrated the 
connection between learning in physics and students' personal epistemologies (Halloun & 
Hestenes, 1998; Hammer & Elby, 2003; Lising & Elby, 2005], Yet this research has typically used 
time intensive qualitative methods or Likert scale surveys which can result in loss of meaning 
(McKeown, 2001], Thus, this study used Q methodology to determine the various perspectives of 
students related to their learning within a first semester, college physics course. 

The use of Likert-scale surveys for a more objective means of assessing epistemological 
beliefs started in the mid-1980's (Ryan, 1984] and has continued to be popular with the 
development of instruments such as the survey developed by Schommer (1990], In this study, 
concourse development started with the items from Schommer's survey and supplemented them 
with statements taken from student interviews (Ramlo, 2006/2007; Ramlo, 2008a; Ramlo, 2008b], 
The Q sample consisted of 30 statements related to learning physics. 
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In this study, first semester college physics students sorted the Q-sample into a grid similar 
to the one shown in Figure 1, after completing the final exam. The condition of instruction was to 
sort the statements based upon their view of their learning in this first semester physics course, 
both lab and lecture. The week before, during the last lab meeting, these same students completed 
the Force and Motion Conceptual Evaluation (FMCE]. They also completed the FMCE during the 
first lab meeting of the semester. The FMCE is frequently used to determine conceptual 
understanding of force and motion (Dykstra, Boyle, & Monarch, 1992; Thornton, 1997; Thornton & 
Sokoloff, 1998] and has been found to be valid and reliable (Ramlo, 2008d], 

Eighteen students completed the Q-sort and the FMCE posttest. Analyses of the Q-sorts 
revealed four factors/views about learning physics in the course. The view represented by those on 
factor 1 contains the largest factor group from the class, seven students. In contrast, factors 2 and 3 
are represented by three participants each. Factor 4 is represented by only one student. Similar 
factor structure was found in prior studies that used the same Q-sample (Ramlo, 2008a]or a Q- 
sample that was very similar (Ramlo, 2006/2007], 


Table 1 - Correlations between the factors (views] and the FMCE posttest scores 


Factor/View 

Correlation 
with Posttest 

score 

Average 
Posttest score 

Standard 

Deviation 

Number of 
students 

1 

.463 

31 

7 

7 

2 

-.393 

16 

11 

3 

3 

-.171 

21 

3 

3 

4 

-.318 

12 

N/A 

1 


Notes: The Force and Motion Conceptual Evaluation (FMCE) was used for the posttest and has a 
maximum of 47points possible. Only Factor 1 had a positive correlation with the posttest. 
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From the correlations shown in Table 1, it is apparent that there is something unique about 
the view represented by Factor 1. This particular view has a high positive correlation (.46] with the 
Force and Motion Conceptual Evaluation [FMCE] posttest scores. The remaining views (factors] all 
have negative correlations with the posttest scores, ranging from -.393 to -.171. The mean FMCE 
posttest score for Factor 1 is 31 +/- 7 whereas the remaining factors' mean FMCE posttest scores 
ranged from 12 to 21. Thus, the correlations indicate that the factor 1 view may be important to 
investigate. 

The results from the analyses for all four factors are given in Table 2. This table contains the 
30 statements from the Q-sample and their resulting grid positions for each of the four 
factors/views. Statement grid positions with an asterisk indicate that these statements were 
distinguishing for that factor when compared to the statement's grid position for each of the other 
factors. 

Table 2 - Statements and their positions for each of the four factors 


Statement 

# 

Statement 

Factor 1 
grid 

position 

Factor 2 
grid 

position 

Factor 3 
grid 

position 

Factor 4 
grid 

position 

1 

I see the ideas of force and motion 
as coherent and interconnected. 

1* 

-4 

-4 

-2 

2 

I think of learning as reconstructing 
and refining my current 
understanding. 

2 

-3 

2 

-3 

3 

When my predictions in lab don't 
match my lab results I question my 
understanding. 

-1 

1 

0 

3 

4 

I like it when my instructor gives me 
the answer instead of making me 
figure it out myself. 

-2 

3 

2 

-4 

5 

I have very little control over how 
much I learn in this course. 

-3 

-1 

-1 

2* 

6 

In lab, if I don't understand 

0 

-2 

0 

1 
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something right away, I will keep on 
trying until I get it. 

Learning something really well 
takes me a long time in this course. 

In this course, if I don't understand 
8 something quickly, it usually means -4* 

I won't understand it. 

Working with classmates helps me 
learn in this course & lab. 


1 * 


0 


2 


-3 


3 


-1 


-1 


1 


-3 
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Table 2 - Statements and their positions for each of the four factors (continued) 


Statement 

# 

Statement 

Factor 1 
grid 

position 

Factor 2 
grid 

position 

Factor 3 
grid 

position 

Factor 4 
grid 

position 

10 

I can tell when I understand the 
material in this class. 

I feel comfortable applying what I 

3 

3 

1 

2 

11 

learned in this class to the real- 
world. 

2 

-1 

-2 

2 

12 

I like the exactness of math-type 
subjects. 

4 

- 2 * 

4 

1 * 

13 

What I learn in this class will help 
me in other classes. 

When I don't understand something 

3 

1 

-2 

-2 

14 

in my physics lab, I try to figure it 
out myself. 

When I don't understand something 

1 

-3 

2 

-1 

15 

in my physics lab, I ask another 
student to help me understand. 

If I am going to understand 

3 

2 

1 

3 

16 

something in this course, it will 
make sense to me right away. 

Sometimes I just have to accept 

-3* 

2 

2 

3 

17 

answers from my professor even 
though I don't understand them. 

- 1 * 

2 

1 

_4* 

18 

What I learn in lab will help me in 
other classes. 

-1 

-2 

4 * 

0 

19 

I am genuinely interested in 
learning about force and motion. 

When I don't understand something 

0 

_4* 

-1 

-1 

20 

in my physics lab, I ask my 
instructor to help me understand. 

1 * 

0 * 

_4* 

4 * 
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Table 2 - Statements and their positions for each of the four factors (continued) 


Statement 

# 

Statement 

Factor 1 
grid 

position 

Factor 2 
grid 

position 

Factor 3 
grid 

position 

Factor 4 
grid 

position 

22 

I try to relate my life experiences to 
the lab activities and/or ideas. 

2 

0 

-3 

0 

23 

Sometimes I found the lab results 
hard to truly believe. 

-4 

-1 

0 

-3 

24 

Sometimes I find I have problems 
understanding the terms used in 
physics. 

-2* 

3* 

1 

1 

25 

My lab results are often consistent 
with my every day thinking about 
how things work. 

1 

0 

-2 

0 


We focus our attention on the grid positions for the Factor 1 statements because this factor 
is of greatest interest. These +/-4 and +/-3 position statements indicate that those represented by 
the Factor 1 view were reflective, help seeking, and enjoyed math / problem solving. Those 
represented by the Factor 1 view also saw the relevance of this physics course and its relationship 
to future classes they would take in their engineering technology major. These students also did not 
see learning as immediate (disagree with statements 7, 8, and 16) but did see that they have control 
over their learning (statement 5). 

To further differentiate this view from the remaining three, we consider the distinguishing 
statements for Factor 1. These statements distinguish this view from the other three views 
statistically. The examination of the distinguishing statements reveals that Factor 1 students sought 
a coherent view of force and motion (statement 1) and believed that their learning would take time 
(statements 8 and 16). It is important to note that only those represented by this view agreed that 
they sought coherence for the force and motion concepts and disagreed that learning needed to be 
immediate (statement 16). 
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Results from this study indicated the need for changes in the lab and course activities for 
this first semester physics course. For instance, although lab activities ask students to reflect on 
earlier, related activities, students disagreed that they try to combine these ideas across the lab 
activities. A need for more effective ways of assisting students in combining these ideas across the 
lab activities is indicated with this finding. The results of this study also indicate the importance of 
helping students seek coherence among the concepts of force and motion. The force and motion 
relationships are stressed throughout the course, including the Realtime Physics Laboratory 
Activities (Sokoloff, Thornton, & Laws, 2004], Yet this study reveals only those that seek this 
coherent view become Newtonian thinkers about force and motion. Assessing students' views 
about learning physics early in the semester may help reveal the need for interventions to assist 
students to become Newtonian thinkers about force and motion. 

Evaluating a New Bioinformatics Course 

Q methodology was used to as part of a program evaluation plan to determine students' 

views of a bioinformatics course created as part of a National Science Foundation grant (Ramlo et 
al., 2008], Bioinformatics is, essentially, the application of computational tools to biological data. 
Thus one of the challenges to creating such an interdisciplinary course was serving the needs and 
backgrounds of a diverse set of students. Primarily these students were a mixture of computer 
science and biology students; students were at both the undergraduate or graduate level. In this 
study, students sorted 29 statements about the course. The two course sections, one spring 2005 
and the other spring 2006, were evaluated separately because of changes made to the 2005 course. 
These changes were based upon the 2005 students' written and verbal feedback. 

The results of the analyses of the Q sorts revealed two factors (views] for each of the course 
sections. Each factor consisted of both computer science and biology students as well as both 
undergraduates and graduate students. Thus, the views that emerged about the course and the 
study of bioinformatics were not similar among those with the same major or student status 
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(undergraduate versus graduate] (Ramlo et al., 2008], This is important to note for program 
evaluation (McNeil, Newman, & Steinhauser, 2005], The Q methodology results indicated that 
changes made to the course after 2005 eliminated some students concerns about their learning, 
primarily related to the computer programming aspects of the course. These course changes also 
appeared to promote a more a positive view of bioinformatics both academically and as a potential 
field of study for the students. This type of information may be helpful for others creating 
bioinformatics courses or programs as well as other inherently interdisciplinary academic 
opportunities (Ramlo et al., 2008], 

Faculty Views about the Creation of a School of Technology 

The investigation of faculty views and consensus regarding the creation of a School of 

Technology at a large Midwestern public university (Ramlo, 2005] demonstrates how Q 
methodology is a powerful tool for determining consensus and perspectives of a group. In this 
study, the administration of the university suggested that a School of Technology be created 
virtually (electronically but not physically] and formed a committee of faculty to investigate this 
conceptualization, including strengths and weaknesses. This type of organizational change in higher 
education can prove difficult and time consuming (Bender & Schuh, 2002], Kezar (2005] suggests 
that successful promotion of organizational change in higher education requires a shared and 
inclusive process. The use of Q methodology allowed the researcher to make this process efficient 
as well as inclusive. Comments from the participants included an appreciation for the democratic 
nature of Q. 

As already mentioned, the participants / committee members statements of strengths and 
weaknesses related to the creation of the School of Technology were used to develop the Q sample 
that they later sorted. Ten committee members sorted the 50 statements of the Q sample. Analysis 
via PQ Method (Schmolck, 2002], a program designed specifically for the analysis of Q sorts using Q 
methodology, revealed three unique views / factors. The tables produced by the Q analyses include 
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a representative sort for each view. It also produces a table of consensus statements (agreement] 
among the factors as well as distinguishing statements (those statements that differentiate each 
factor/ view from the others]. 

Half of the sorters (5] were represented by factor 1 which possessed a positive view about 
the creation of the School of Technology. The representative sort for this view as well as the 
distinguishing statements revealed that this view was extremely positive about the creation of the 
School of Technology. They were not concerned about details such as funding, location, or name. 
Instead, this view believed that the creation of the School of Technology would improve programs' 
images, promote their bachelor degrees, and encourage innovation. 

The second factor / view produced was bipolar with two loaders. Thus, persons 1 and 2 on 
this factor have inverted sorts or, in other words, opposing views. One loader has a positive factor 
score and the other a negative factor score. Conceptually this is like having one person positively 
correlated with the factor and the other negatively correlated. The positive loader was cautious 
about the creation of the School of Technology because of the need for additional resources that 
may not be included and because the proposed organization is not similar to other schools of 
technology at other institutions. The negative loader has an inverted view; in other words this 
person is not concerned about resources and believes that it is good that the School of Technology 
is not organized like those at other universities. The remaining view represented only one sorter. 
This view saw the creation of the School of Technology as a marketing opportunity for the pre¬ 
existing bachelor degrees that would be associated with the school. This view also saw students as a 
key component to the choices made about the school as it was organized. 

The six consensus statements revealed that all three views agreed that the organization 
would need to be flexible with constant evaluation and assessment such that changes could be 
made to improve the School of Technology. Other shared concerns included the ability to market 
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the School of Technology and its program separately from the university and the potential of an 
increase in revenue from the programs not being fed back into the School of Technology. 

The revealing the differing views within a group is important for collaboration (Clark et al., 
1996], In this study, not only did Q reveal the differing views in detail, but it also did this in 
a efficient manner; this is also important for the type of assessment used to promote 
organizational change (Bender & Schuh, 2002). Q also determined consensus items which 
also promoted discussion and collaboration. This consensus, along with the determining of 
different views, promoted the type of facilitation suggested by Witte and Engelhardt 
(2004). These findings were supported by committee members’ comments about the 
empowering and democratic feelings they had about the study and its results. 


Conclusions 

The findings from this study are unique compared to other studies that investigated 
students' epistemological views related to learning physics and their conceptual understanding. 
Unlike the CLASS (Adams, Perkins, Dubson, Finkelstein, & Wieman, 2005; Perkins, Adams, Pollock, 
Finkelstein, & Wieman, 2005; Perkins, Gratny, Adams, Finkelstein, & Wieman, 2006) and VASS 
(Halloun & Hestenes, 1996; Halloun & Hestenes, 1998) surveys, this Q methodology study revealed 
a variety of views about learning physics. Unlike the CLASS studies, students' views were not simply 
measured relative to the views of selected experts (Adams et al., 2005; Perkins et al., 2005; Perkins 
et al., 2006). Instead, four unique views were determined from students' sorts that revealed a richer 
picture of the beliefs about learning physics within the course, including the inquiry laboratories. 
These perspectives led to modifications within the laboratories and course. Evaluation of students' 
views of learning physics is ongoing for students enrolled within this first semester physics course. 
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The bioinformatics study demonstrated that classifying individuals into groups is helpful in 
various research situations including program evaluation. Using Q methodology to group people 
based upon their perspectives, using factor analysis to correlate their views as expressed by their Q 
sorts, is a more effective method of grouping people than using surface characteristics such as race, 
sex or academic major because surface characteristics do not necessarily determine similar views / 
perspectives. This is frequently important in program evaluation, where there is often value in 
addressing the various stakeholder groups differently to ascertain their needs and make effective 
program improvements (McNeil et al., 2005], In addition, the use of Q methodology also effectively 
reduces the huge amount of qualitative variables into groupings to better investigate research 
questions. Thus, it behooves education researchers to learn more about Q methodology and learn 
about the types of research purposes that Q can be used to address. 

Within the School of Technology study, Q methodology allowed the researcher to promote the 
type of assessment that Bender (2002) describes as optimal for organizational change. She specifically 
stated that organizational change assessment needs to be effective in that the appropriate issues must be 
identified, the right questions need to be asked, and collection and analysis of data needs to be efficient. Q 
methodology demonstrated each of these traits for assessment within this study. The Q results also 
allowed committee members to feel empowered and that everyone’s voice was heard; these feelings are 
important for effective collaboration (Clark et al., 1996). 

In summary, the studies discussed here demonstrate that Q methodology is a powerful tool for 
determining personal perspectives within higher education. In two of the studies, students’ views were 
determined and then used to make course changes to improve students’ learning and attitudes toward the 
subject. In the third study, faculty collaboration was improved and ideas about organizational change 
were revealed via the use of Q methodology. Thus, these studies demonstrate the versatility of Q 
methodology in addressing research purposes in higher education and the importance of determining the 
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variety of views that exist about a topic, whether that topic is organizational change, a newly developed 
bioinformatics course, or learning in a first semester college physics course. 
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